ˆ f. Contents. Overview. Function Approximation. f ˆ : X Y. y x m. Introduction to Radial Basis Function Networks RBF
|
|
- Kathryn Barton
- 6 years ago
- Views:
Transcription
1 Introducton to Radal Bass Functon Networks Contents Overvew he Models of Functon Aroator he Radal Bass Functon Networks RBFN s for Functon Aroaton he Proecton Matr Learnng the Kernels Bas-Varance Dlea he Effectve Nuber of Paraeters Model Selecton Introducton to Radal Bass Functon Networks Overvew RBF Lnear odels have been studed n statstcs for about 00 ears and the theor s alcable to RBF networks whch are ust one artcular te of lnear odel. However, the fashon for neural networks whch started n the d-80 has gven rse to new naes for concets alread falar to statstcans cal Alcatons of NN Functon Aroaton Pattern Classfcaton X R l f () lc N Functon Aroaton X R f () YR e-seres Forecastng ( ) (,,, ) t f t t t3 n X R Unknown Aroator X R f : X Y f f ˆ : X Y ˆ f Y R n Y R n
2 Suervsed Learnng Neural Networks as Unversal Aroators,, Unknown Functon Neural Network ˆ + + e Feedforward neural networks wth a sngle hdden laer of sgodal unts are caable of aroatng unforl an contnuous ultvarate functon, to an desred degree of accurac. Lke feedforward neural networks wth a sngle hdden laer of sgodal unts, t can be shown that RBF networks are unversal aroators. Statstcs vs. Neural Networks Introducton to Radal Bass Functon Networks Statstcs Model Estaton Regresson Interolaton Observatons Paraeters Indeendent varables Deendent varables Rdge regresson Neural Networks Network Learnng Suervsed learnng Generalzaton ranng set (snatc) Weghts Inuts Oututs Weght deca he Model of Functon Aroator Lnear Models Lnear Models f () w ( ) f Weghts () w ( ) Fed Bass Functons Hdden Unts Inuts Outut Unts Lnearl weghted outut w w w = n Decooston Feature Etracton ransforaton Feature Vectors
3 Lnear Models Hdden Unts Inuts = Outut Unts Lnearl weghted outut w w w n f () w ( ) Decooston Feature Etracton ransforaton Feature Vectors Eale Lnear Models Polnoal f () w Fourer Seres f () w e k k k ( ), 0,,, k k0 k 0 f () w ( ) ( ) e, 0,,, Sngle-Laer Percetrons as Unversal Aroators f () w ( ) Radal Bass Functon Networks as Unversal Aroators f () w ( ) Hdden Unts w w w Wth suffcent nuber of sgodal unts, t can be a unversal aroator. Hdden Unts w w w Wth suffcent nuber of radal-bass-functon unts, t can also be a unversal aroator. = n = n Non-Lnear Models f () w ( ) Introducton to Radal Bass Functon Networks f Weghts () w ( ) Adusted b the Learnng rocess he Radal Bass Functon Networks 3
4 f () w ( ) Radal Bass Functons hree araeters for a radal functon: Center Dstance Measure Shae r = ()= ( ) cal Radal Functons Gaussan r r e 0 and r Hard-Multquadratc (97) r r c c c 0 and r Inverse Multquadratc r c r c c 0 and r r r e 0 and r Gaussan Bass Functon (=0.5,.0,.5) r c r c c 0 and r Inverse Multquadratc c=5 c=4 c=3 c= c= Bass { : =,, } s `near orthogonal. Most General RBF ( μ) ( μ) μ μ μ 3 Proertes of RBF s On-Center, Off Surround Analoges wth localzed recetve felds found n several bologcal structures, e.g., vsual corte; ganglon cells 4
5 As a functon f() aroator he oolog of RBF As a attern classfer. he oolog of RBF Outut Unts Interolaton Outut Unts Classes Hdden Unts Proecton Hdden Unts Subclasses Inuts n Feature Vectors Inuts n Feature Vectors Introducton to Radal Bass Functon Networks RBFN s for Functon Aroaton Radal Bass Functon Networks Radal bass functon (RBF) networks are feedforward networks traned usng a suervsed tranng algorth. he actvaton functon s selected fro a class of functons called bass functons. he usuall tran uch faster than BP. he are less suscetble to robles wth nonstatonar nuts Radal Bass Functon Networks he dea Poularzed b Broohead and Lowe (988), and Mood and Darken (989), RBF networks have roven to be a useful neural network archtecture. he aor dfference between RBF and BP s the behavor of the sngle hdden laer. Rather than usng the sgodal or S-shaed actvaton functon as n BP, the hdden unts n RBF networks use a Gaussan or soe other bass kernel functon. Unknown Functon to Aroate ranng Data 5
6 he dea he dea w f( ) ( ) Unknown Functon to Aroate Functon Learned ranng Data Bass Functons (Kernels) Bass Functons (Kernels) he dea w f( ) ( ) he dea w f( ) ( ) Nontranng Sale Functon Learned Nontranng Sale Functon Learned Bass Functons (Kernels) Radal Bass Functon Networks as Unversal Aroators ranng set, Goal ( k ) ( k f ) for all k n SSE k f k w k f ( ) w ( ) w w w = n Learn the Otal Weght Vector ranng set, Goal ( k ) ( k f ) for all k n SSE k f k w k f ( ) w ( ) w w w = n 6
7 Regularzaton ranng set, Goal ( k ) ( k f ) for all k n SSE C k f w k k w w 0 If regularzaton s unneeded, set 0 Learn the Otal Weght Vector Mnze C w f ( ) ( ) w f k C f 0 w f w k w k k ( ) ( ) f w k f w ( ) w ( ) k k f w f ( ) ( ) ( ) w ( ) Learn the Otal Weght Vector f φ f w φ,, Learn the Otal Weght Vector φ f w φ,, Defne () ( ),, f,, f () ( ),, φ f () ( ) f w k k φ f w φ w φ f φ f w φ φ Defne Φ f Λw Φ φ, φ,, φ Λ w w, w,, w Φ Learn the Otal Weght Vector () () wk k () () () f k () () w f wk k f () () () w k ( ) f ( ) ( ) ( ) w ( P) wk k k Φ Φw Λw Φ f Λw f Φ Φ ( ) w ( ) Φw Φ Φw Λw Φ Learn the Otal Weght Vector Φ ΦΛ w Λ Φ w Φ Φ Φ A Φ Φ: A : Desgn Matr Varance Matr 7
8 Introducton to Radal Bass Functon Networks w A Φ he Ercal-Error Vector f Φw he Proecton Matr Unknown Functon w w φ φ w φ n n w A Φ he Ercal-Error Vector f Φw Error Vector w w w Unknown φ φ e f Φw ΦA Functon I ΦA Φ P n Φ φ n Su-Squared-Error Error Vector A ΦΦΛ e P SSE f k If =0, the RBFN s learnng algorth s to nze SSE (MSE). Φ φ, φ,, φ PI ΦA Φ k ( ) P P P he Proecton Matr Error Vector SSE P A ΦΦΛ e P Λ0 Φ φ, φ,, φ PI ΦA Φ e san ( φ, φ,, φ) P( P) Pe e P P Introducton to Radal Bass Functon Networks Learnng the Kernels 8
9 RBFN s as Unversal Aroators What to Learn? l w w w w l w l w ll ranng set, k Kernels μ ( ) e l w w w w l w l w ll Weghts w s Centers s of s Wdths s of s Nuber of s Model Selecton n n l f w One-Stage Learnng ( k ) ( k ) ( k ) w f l μ w f μ μ l w f 3 3 l f w he sultaneous udates of all three sets of araeters a be sutable One-Stage Learnng for non-statonar envronents or onlne settng. ( k ) ( k ) ( k ) w f l μ w f μ μ l w f 3 3 wo-stage ranng ran the Kernels l Ste Deternes w s. w w w w l w l w ll E.g., usng batch-learnng. n Ste Deternes Centers s of s. Wdths s of s. Nuber of s. 9
10 Unsuervsed ranng Methods Subset Selecton Rando Subset Selecton Forward Selecton Backward Elnaton Clusterng Algorths KMEANS LVQ Mture Models GMM Subset Selecton Rando Subset Selecton Randol choosng a subset of onts fro tranng set Senstve to the ntall chosen onts. Usng soe adatve technques to tune Centers Wdths #onts Clusterng Algorths Clusterng Algorths 0
11 Clusterng Algorths Clusterng Algorths 4 3 Introducton to Radal Bass Functon Networks Bas-Varance Dlea Goal Revst Ultate Goal Generalzaton Mnze Predcton Error Goal of Our Learnng Procedure Mnze Ercal Error Badness of Ft Underfttng A odel (e.g., network) that s not suffcentl cole can fal to detect full the sgnal n a colcated data set, leadng to underfttng. Produces ecessve bas n the oututs. Overfttng A odel (e.g., network) that s too cole a ft the nose, not ust the sgnal, leadng to overfttng. Produces ecessve varance n the oututs. Underfttng/Overfttng Avodance Model selecton Jtterng Earl stong Weght deca Regularzaton Rdge Regresson Baesan learnng Cobnng networks
12 Best Wa to Avod Overfttng Badness of Ft Use lots of tranng data, e.g., 30 tes as an tranng cases as there are weghts n the network. for nose-free data, 5 tes as an tranng cases as weghts a be suffcent. Don t arbtrarl reduce the nuber of weghts for fear of underfttng. Underft Overft Badness of Ft However, t's not reall a dlea. Bas-Varance Dlea Underft Overft Underft Overft Large bas Sall varance Sall bas Large varance It's not reall a dlea. Bas-Varance Dlea Bas-Varance Dlea More on overfttng Underft Easl lead to redctons that are far beond the range of the tranng data. Produce wld redctons n ultlaer ercetrons even Large wth nose-free bas data. Sall varance Overft Sall bas Large varance ore underft Bas Varance ft ore overft
13 he ean of the bas=? he varance of the bas=? Bas-Varance Dlea he ean of the bas=? he varance of the bas=? Bas-Varance Dlea nose bas bas bas he true odel Varance nose bas he true odel Soluton obtaned wth tranng set. Soluton obtaned wth tranng set 3. Soluton obtaned wth tranng set. Sets of functons E.g., deend on # hdden nodes used. Sets of functons E.g., deend on # hdden nodes used. Reduce the effectve nuber of araeters. Reduce the nuber of hdden nodes. Model Selecton Goal: n E ( ) f ( ) Bas-Varance Dlea Varance nose bas he true odel ( ) g( ) nose he true odel g() bas f () Sets of functons E.g., deend on # hdden nodes used. Sets of functons E.g., deend on # hdden nodes used. Goal: n E ( ) f ( ) Bas-Varance Dlea Goal: n E ( ) f ( ) n Eg( ) f ( ) ( ) ( ) E g( ) f ( ) E f ( ) ( ) ( ) ( ) E g f g f ( ) ( ) ( ) ( ) E g f E g f E 0 constant ( ) ( ) ( ) ( ) E f E E g f Bas-Varance Dlea Eg( ) f ( ) E g( ) E[ f ( )] E[ f ( )] f ( ) Eg( ) E[ f ( )] f ( ) E[ f ( )] g( ) E[ f ( )] f ( ) E[ f ( )] Eg( ) Ef [ ( )] E f( ) Ef [ ( )] E g( ) E[ f ( )] f ( ) E[ f ( ) ] 0 E g( ) E[ f ( )] f( ) E[ f ( )] Eg( ) f ( ) Eg( )E[ f ( )] EEf [ ( )] f ( ) EE[ f ( )] E[ f ( )] Eg( ) E f ( ) Eg( ) E[ f ( )] Ef [ ( )] E f () E[ f ( )] E[ f ( )] 0 3
14 Goal: n E ( ) f ( ) Bas-Varance Dlea ( ) ( ) ( ) ( ) E f E E g f E Eg( ) Ef [ ( )] E f ( ) Ef [ ( )] nose bas varance Goal: n E ( ) f ( ) Model Colet vs. Bas-Varance ( ) ( ) ( ) ( ) E f E E g f E Eg( ) Ef [ ( )] E f ( ) Ef [ ( )] nose bas varance Cannot be nzed Mnze both bas and varance Model Colet (Caact) Goal: n E ( ) f ( ) Bas-Varance Dlea ( ) ( ) ( ) ( ) E f E E g f E Eg( ) Ef [ ( )] E f ( ) Ef [ ( )] Eale (Polnoal Fts) e( 6 ) nose bas varance Model Colet (Caact) Eale (Polnoal Fts) Eale (Polnoal Fts) Degree Degree 5 Degree 0 Degree 5 4
15 Introducton to Radal Bass Functon Networks he Effectve Nuber of Paraeters Varance Estaton Mean ˆ In general, not avalable. Varance Varance Estaton Sle Lnear Regresson Mean ˆ ˆ s ˆ Varance 0 ~ N(0, ) 0 Loss degree of freedo Mnze ˆ Sle Lnear Regresson SSE Mnze ˆ Mean Squared Error (MSE) SSE 0 ~ N(0, ) ŷ ˆ ˆ 0 0 ~ N(0, ) ŷ ˆ ˆ 0 ˆ SSE MSE Loss degrees of freedo 5
16 Varance Estaton SSE ˆ MSE Loss degrees of freedo : #araeters of the odel he Nuber of Paraeters f ( ) w ( ) w w w = n #degrees of freedo: he Effectve Nuber of Paraeters () Facts: trace ( AB) trace ( A) trace ( B) trace ( AB) trace ( BA) he Effectve Nuber of Paraeters () f ( ) w ( ) he roecton Matr w w w = n PI ΦA Φ A ΦΦΛ Λ 0 trace( P) Pf) trace ( ) P trace I ΦA Φ trace ΦA Φ trace Φ Φ Φ Φ trace Φ Φ Φ Φ trace I he roecton Matr PI ΦA Φ A ΦΦΛ Λ 0 trace( P) he effectve nuber of araeters: trace( P) Regularzaton he effectve nuber of araeters: trace ( P) Regularzaton Cost Eral Error f k w SSE k Penalze odels wth large weghts Model s enal t w Cost Eral Error f k w SSE k Penalze odels wth large weghts Wthout enalt ( Model =0), there are degrees s of freedo to nze enal SSE t(cost). he effectve nuber of araeters =. w 6
17 he effectve nuber of araeters: ( ) he effectve nuber of araeters: trace ( P) trace P Regularzaton Varance Estaton Cost Eral Error f k w SSE k Penalze odels wth large weghts Model s enal t Wth enalt ( >0), the lbert to nze SSE wll be reduced. he effectve nuber of araeters <. w ˆ MSE SSE Loss degrees of freedo Varance Estaton ˆ he effectve nuber of araeters: trace ( P) SSE MSE trace ( P) Introducton to Radal Bass Functon Networks Model Selecton Model Selecton Goal Choose the fttest odel Crtera Least redcton error Man ools (Estate Model Ftness) Cross valdaton Proecton atr Methods Weght deca (Rdge regresson) Prunng and Growng RBFN s Ercal Error vs. Model Ftness Ultate Goal Generalzaton Mnze Predcton Error Goal of Our Learnng Procedure Mnze Ercal Error (MSE) Mnze Predcton Error 7
18 Estatng Predcton Error When ou have lent of data use ndeendent test sets E.g., use the sae tranng set to tran dfferent odels, and choose the best odel b coarng on the test set. When data s scarce, use Cross-Valdaton Bootstra Cross Valdaton Slest and ost wdel used ethod for estatng redcton error. Partton the orgnal set nto several dfferent was and to coute an average score over the dfferent arttons, e.g., K-fold Cross-Valdaton Leave-One-Out Cross-Valdaton Generalze Cross-Valdaton K-Fold CV Slt the set, sa, D of avalable nut-outut atterns nto k utuall eclusve subsets, sa D, D,, D k. K-Fold CV Avalable Data ran and test the learnng algorth k tes, each te t s traned on D\D and tested on D. K-Fold CV est Set ranng Set A secal case of k-fold CV. Leave-One-Out CV D D D 3 Avalable. Data.. D k D D D 3... D k D D D 3... D k D D D 3... D k D D D 3... D k Estate Slt the avalable nut-outut atterns nto a tranng set of sze and a test set of sze. Average the squared error on the left-out attern over the ossble was of artton. 8
19 A secal case of k-fold CV. Error Varance Predcted b LOO D (, ): k,,, k D D\ (, ) f k,, Functon learned usng D as tranng set. Avalable nut-outut atterns. ranng sets of LOO. he estate for the varance of redcton error usng LOO: ˆ ( ) LOO f Error-square for the left-out eleent. A secal case of k-fold CV. Error Varance Gven a odel, Predcted the functon wth b least LOO D (, ): k,,, k D D\ (, ) f k ercal error for D.,, Functon learned usng D as tranng set. Avalable nut-outut atterns. ranng sets of LOO. As an nde of odel s ftness. he estate We want for to fnd the a varance odel also of redcton nze ths. error usng LOO: ˆ ( ) LOO f Error-square for the left-out eleent. A secal case of k-fold CV. Error Varance Predcted b LOO D (, ): k,,, k D D\ (, ) f k,, Functon learned usng D as tranng set. Avalable nut-outut atterns. ranng sets of LOO. he estate for the varance of redcton error usng LOO: ˆ ( ) LOO f Error-square for the left-out eleent. Error Varance Predcted b LOO ˆLOO ˆ P( dag( P)) Pˆ ˆ ( ) LOO f Error-square for the left-out eleent. ˆ LOO ˆ P( dag( P)) Pˆ Generalzed Cross-Valdaton More Crtera Based on CV trace ( P) dag ( P) I ˆ GCV ˆ ˆ trace ( P) P ˆ ˆ ( ) P ˆ GCV ˆ ˆ ( ) P ˆ P ˆ ˆUEV ˆ ˆ ˆFPE GCV (Generalzed CV) UEV (Unbased estate of varance) P FPE ln( ) ˆ P ˆ ˆ BIC (Fnal Predcton Error) Akake s Inforaton Crteron BIC (Baesan Inforaton Crtero) 9
20 ˆ ˆ ˆ ˆ UEV FPE GCV BIC More Crtera Based on CV ˆ ˆ ˆ ˆ UEV FPE GCV BIC More Crtera Based on CV ˆ GCV ˆ ˆ ( ) P ˆ P ˆ ˆUEV ˆ P ˆ ˆFPE ˆUEV ln( ) ˆ P ˆ ˆ BIC ˆUEV ln( ) ˆUEV ˆ GCV ˆ ˆ ( ) P ˆ P ˆ ˆUEV ˆ P ˆ ˆFPE ˆUEV ln( ) ˆ P ˆ ˆ BIC ˆUEV P? ln( ) ˆUEV Standard Rdge Regresson, Regularzaton, Standard Rdge Regresson, Regularzaton, SSE Penalze odels wth large weghts SSE Penalze odels wth large weghts Cost Eral Error f k w k Model s enal t w Cost Eral Error f k w k Model s enal t w w n Soluton Revew w A Φ A Φ ΦI C w w k PI ΦA Φ Used to coute odel selecton crtera Eale Wdth of RBF r = 0.5 ( ) ( ) e ~ N(0,0. ) 50 0
21 Eale Wdth of RBF r = 0.5 ( ) ( ) e ~ N(0,0. ) 50 ˆ ˆ ˆ ˆ UEV FPE GCV BIC Eale Wdth of RBF r = 0.5 ( ) ( ) e ~ N(0,0. ) 50 ˆ ˆ ˆ ˆ UEV FPE GCV BIC Otzng the Regularzaton Paraeter Re-Estaton Forula P trace A ˆ A ˆ w A wtrace ( P) Local Rdge Regresson Re-Estaton Forula P trace A ˆ A ˆ w A wtrace ( P) Eale trace( P) sn( ) 50 r 0.05 Wdth of RBF ~ N(0,0. ) Eale trace( P) sn( ) 50 r 0.05 Wdth of RBF ~ N(0,0. )
22 Eale sn( ) 50 r 0.05 Wdth of RBF ~ N(0,0. ) Eale sn( ) 50 r 0.05 Wdth of RBF ~ N(0,0. ) P trace A ˆ A ˆ w A wtrace ( P) here are two local-na. P trace A ˆ A ˆ w A wtrace ( P) here are two local-na. Usng the about re-estaton forula, t wll be stuck at the nearest local nu. ˆ(0) 0.0 hat s, the soluton deends on the ntal settng. l ˆ( t ) 0. t Eale P trace A ˆ A ˆ w A wtrace ( P) ˆ(0) 0 5 sn( ) 50 r 0.05 Wdth of RBF ~ N(0,0. ) here are two local-na. Eale RMSE: Root Mean Squared Error In real case, t s not avalable. sn( ) 50 r 0.05 Wdth of RBF ~ N(0,0. ) l ˆ() t. 0 t 4 Eale sn( ) 50 r 0.05 Wdth of RBF ~ N(0,0. ) RMSE: Root Mean Squared Error In real case, t s not avalable. Local Rdge Regresson Standard Rdge Regresson n n C w w k Local Rdge Regresson C w w k
23 Local Rdge Regresson Standard Rdge Regresson n C les that () can be reoved. w w k Local Rdge Regresson n C w w k he Solutons w A Φ Lnear Regresson A Φ Φ A Φ ΦI A Φ ΦΛ PI ΦA Φ Used to coute odel selecton crtera Standard Rdge Regresson Local Rdge Regresson ˆ P ˆ ˆ GCV Otzng the trace ( P) Regularzaton Paraeters Otzng the Regularzaton Paraeters Increental Oeraton P: he current roecton Matr. P : he roecton Matr obtaned b reovng (). Solve ˆGCV 0 Subect to 0 ˆ GCV ˆ ˆ trace ( P) P PP P φ φ P φ P φ PP P φ φ P φ P φ Otzng the Regularzaton Paraeters Otzng the Regularzaton Paraeters Solve ˆGCV 0 Subect to 0 a P b Pφ Pφ ( ) c φ P φ Pφ trace ( P ) φ P φ f a b cb ˆ f φ Pφ ba and a b cb 0 f φ Pφ ba and a b c b c b b a φ Pφ f φpφ b a Solve ˆGCV 0 Subect to 0 a P b Pφ Pφ ( ) Reove () c φ P φ Pφ trace ( P ) φ P φ f a b cb ˆ f φ Pφ ba and a b cb 0 f φ Pφ ba and a b c b c b b a φ Pφ f φpφ b a 3
24 he Algorth Intalze s. e.g., erforng standard rdge regresson. Reeat the followng untl GCV converges: Randol select and coute Perfor local rdge regresson If GCV reduce & reove () ˆ ˆ 4
Source-Channel-Sink Some questions
Source-Channel-Snk Soe questons Source Channel Snk Aount of Inforaton avalable Source Entro Generall nos and a be te varng Introduces error and lts the rate at whch data can be transferred ow uch nforaton
More informationMultilayer Perceptrons and Backpropagation. Perceptrons. Recap: Perceptrons. Informatics 1 CG: Lecture 6. Mirella Lapata
Multlayer Perceptrons and Informatcs CG: Lecture 6 Mrella Lapata School of Informatcs Unversty of Ednburgh mlap@nf.ed.ac.uk Readng: Kevn Gurney s Introducton to Neural Networks, Chapters 5 6.5 January,
More informationCOMP th April, 2007 Clement Pang
COMP 540 12 th Aprl, 2007 Cleent Pang Boostng Cobnng weak classers Fts an Addtve Model Is essentally Forward Stagewse Addtve Modelng wth Exponental Loss Loss Functons Classcaton: Msclasscaton, Exponental,
More informationDiscrete Memoryless Channels
Dscrete Meorless Channels Source Channel Snk Aount of Inforaton avalable Source Entro Generall nos, dstorted and a be te varng ow uch nforaton s receved? ow uch s lost? Introduces error and lts the rate
More informationGradient Descent Learning and Backpropagation
Artfcal Neural Networks (art 2) Chrstan Jacob Gradent Descent Learnng and Backpropagaton CSC 533 Wnter 200 Learnng by Gradent Descent Defnton of the Learnng roble Let us start wth the sple case of lnear
More informationExcess Error, Approximation Error, and Estimation Error
E0 370 Statstcal Learnng Theory Lecture 10 Sep 15, 011 Excess Error, Approxaton Error, and Estaton Error Lecturer: Shvan Agarwal Scrbe: Shvan Agarwal 1 Introducton So far, we have consdered the fnte saple
More informationClassification (klasifikácia) Feedforward Multi-Layer Perceptron (Dopredná viacvrstvová sieť) 14/11/2016. Perceptron (Frank Rosenblatt, 1957)
4//06 IAI: Lecture 09 Feedforard Mult-Layer Percetron (Doredná vacvrstvová seť) Lubca Benuskova AIMA 3rd ed. Ch. 8.6.4 8.7.5 Classfcaton (klasfkáca) In machne learnng and statstcs, classfcaton s the roblem
More informationLECTURE :FACTOR ANALYSIS
LCUR :FACOR ANALYSIS Rta Osadchy Based on Lecture Notes by A. Ng Motvaton Dstrbuton coes fro MoG Have suffcent aount of data: >>n denson Use M to ft Mture of Gaussans nu. of tranng ponts If
More informationEE513 Audio Signals and Systems. Statistical Pattern Classification Kevin D. Donohue Electrical and Computer Engineering University of Kentucky
EE53 Audo Sgnals and Systes Statstcal Pattern Classfcaton Kevn D. Donohue Electrcal and Couter Engneerng Unversty of Kentucy Interretaton of Audtory Scenes Huan erceton and cognton greatly eceeds any couter-based
More information10-701/ Machine Learning, Fall 2005 Homework 3
10-701/15-781 Machne Learnng, Fall 2005 Homework 3 Out: 10/20/05 Due: begnnng of the class 11/01/05 Instructons Contact questons-10701@autonlaborg for queston Problem 1 Regresson and Cross-valdaton [40
More informationPGM Learning Tasks and Metrics
Probablstc Graphcal odels Learnng Overvew PG Learnng Tasks and etrcs Learnng doan epert True dstrbuton P* aybe correspondng to a PG * dataset of nstances D{d],...d]} sapled fro P* elctaton Network Learnng
More informationGeneralized Linear Methods
Generalzed Lnear Methods 1 Introducton In the Ensemble Methods the general dea s that usng a combnaton of several weak learner one could make a better learner. More formally, assume that we have a set
More information15-381: Artificial Intelligence. Regression and cross validation
15-381: Artfcal Intellgence Regresson and cross valdaton Where e are Inputs Densty Estmator Probablty Inputs Classfer Predct category Inputs Regressor Predct real no. Today Lnear regresson Gven an nput
More informationSupport Vector Machines. Vibhav Gogate The University of Texas at dallas
Support Vector Machnes Vbhav Gogate he Unversty of exas at dallas What We have Learned So Far? 1. Decson rees. Naïve Bayes 3. Lnear Regresson 4. Logstc Regresson 5. Perceptron 6. Neural networks 7. K-Nearest
More informationWhich Separator? Spring 1
Whch Separator? 6.034 - Sprng 1 Whch Separator? Mamze the margn to closest ponts 6.034 - Sprng Whch Separator? Mamze the margn to closest ponts 6.034 - Sprng 3 Margn of a pont " # y (w $ + b) proportonal
More informationDeparture Process from a M/M/m/ Queue
Dearture rocess fro a M/M// Queue Q - (-) Q Q3 Q4 (-) Knowledge of the nature of the dearture rocess fro a queue would be useful as we can then use t to analyze sle cases of queueng networs as shown. The
More informationy new = M x old Feature Selection: Linear Transformations Constraint Optimization (insertion)
Feature Selecton: Lnear ransforatons new = M x old Constrant Optzaton (nserton) 3 Proble: Gven an objectve functon f(x) to be optzed and let constrants be gven b h k (x)=c k, ovng constants to the left,
More informationBAYESIAN CURVE FITTING USING PIECEWISE POLYNOMIALS. Dariusz Biskup
BAYESIAN CURVE FITTING USING PIECEWISE POLYNOMIALS Darusz Bskup 1. Introducton The paper presents a nonparaetrc procedure for estaton of an unknown functon f n the regresson odel y = f x + ε = N. (1) (
More informationINF 5860 Machine learning for image classification. Lecture 3 : Image classification and regression part II Anne Solberg January 31, 2018
INF 5860 Machne learnng for mage classfcaton Lecture 3 : Image classfcaton and regresson part II Anne Solberg January 3, 08 Today s topcs Multclass logstc regresson and softma Regularzaton Image classfcaton
More information1 Definition of Rademacher Complexity
COS 511: Theoretcal Machne Learnng Lecturer: Rob Schapre Lecture #9 Scrbe: Josh Chen March 5, 2013 We ve spent the past few classes provng bounds on the generalzaton error of PAClearnng algorths for the
More informationIntro to Visual Recognition
CS 2770: Computer Vson Intro to Vsual Recognton Prof. Adrana Kovashka Unversty of Pttsburgh February 13, 2018 Plan for today What s recognton? a.k.a. classfcaton, categorzaton Support vector machnes Separable
More informationFor now, let us focus on a specific model of neurons. These are simplified from reality but can achieve remarkable results.
Neural Networks : Dervaton compled by Alvn Wan from Professor Jtendra Malk s lecture Ths type of computaton s called deep learnng and s the most popular method for many problems, such as computer vson
More informationMACHINE APPLIED MACHINE LEARNING LEARNING. Gaussian Mixture Regression
11 MACHINE APPLIED MACHINE LEARNING LEARNING MACHINE LEARNING Gaussan Mture Regresson 22 MACHINE APPLIED MACHINE LEARNING LEARNING Bref summary of last week s lecture 33 MACHINE APPLIED MACHINE LEARNING
More informationDiscriminative classifier: Logistic Regression. CS534-Machine Learning
Dscrmnatve classfer: Logstc Regresson CS534-Machne Learnng 2 Logstc Regresson Gven tranng set D stc regresson learns the condtonal dstrbuton We ll assume onl to classes and a parametrc form for here s
More informationSupporting Information
Supportng Informaton The neural network f n Eq. 1 s gven by: f x l = ReLU W atom x l + b atom, 2 where ReLU s the element-wse rectfed lnear unt, 21.e., ReLUx = max0, x, W atom R d d s the weght matrx to
More informationPROBABILITY AND STATISTICS Vol. III - Analysis of Variance and Analysis of Covariance - V. Nollau ANALYSIS OF VARIANCE AND ANALYSIS OF COVARIANCE
ANALYSIS OF VARIANCE AND ANALYSIS OF COVARIANCE V. Nollau Insttute of Matheatcal Stochastcs, Techncal Unversty of Dresden, Gerany Keywords: Analyss of varance, least squares ethod, odels wth fxed effects,
More informationWeek 5: Neural Networks
Week 5: Neural Networks Instructor: Sergey Levne Neural Networks Summary In the prevous lecture, we saw how we can construct neural networks by extendng logstc regresson. Neural networks consst of multple
More informationKristin P. Bennett. Rensselaer Polytechnic Institute
Support Vector Machnes and Other Kernel Methods Krstn P. Bennett Mathematcal Scences Department Rensselaer Polytechnc Insttute Support Vector Machnes (SVM) A methodology for nference based on Statstcal
More informationConfidence intervals for weighted polynomial calibrations
Confdence ntervals for weghted olynomal calbratons Sergey Maltsev, Amersand Ltd., Moscow, Russa; ur Kalambet, Amersand Internatonal, Inc., Beachwood, OH e-mal: kalambet@amersand-ntl.com htt://www.chromandsec.com
More informationLecture 19 of 42. MAP and MLE continued, Minimum Description Length (MDL)
Lecture 19 of 4 MA and MLE contnued, Mnu Descrpton Length (MDL) Wednesday, 8 February 007 Wlla H. Hsu, KSU http://www.kddresearch.org Readngs for next class: Chapter 5, Mtchell Lecture Outlne Read Sectons
More informationSystem in Weibull Distribution
Internatonal Matheatcal Foru 4 9 no. 9 94-95 Relablty Equvalence Factors of a Seres-Parallel Syste n Webull Dstrbuton M. A. El-Dacese Matheatcs Departent Faculty of Scence Tanta Unversty Tanta Egypt eldacese@yahoo.co
More informationLinear Classification, SVMs and Nearest Neighbors
1 CSE 473 Lecture 25 (Chapter 18) Lnear Classfcaton, SVMs and Nearest Neghbors CSE AI faculty + Chrs Bshop, Dan Klen, Stuart Russell, Andrew Moore Motvaton: Face Detecton How do we buld a classfer to dstngush
More informationMLE and Bayesian Estimation. Jie Tang Department of Computer Science & Technology Tsinghua University 2012
MLE and Bayesan Estmaton Je Tang Department of Computer Scence & Technology Tsnghua Unversty 01 1 Lnear Regresson? As the frst step, we need to decde how we re gong to represent the functon f. One example:
More informationNeural Networks. Perceptrons and Backpropagation. Silke Bussen-Heyen. 5th of Novemeber Universität Bremen Fachbereich 3. Neural Networks 1 / 17
Neural Networks Perceptrons and Backpropagaton Slke Bussen-Heyen Unverstät Bremen Fachberech 3 5th of Novemeber 2012 Neural Networks 1 / 17 Contents 1 Introducton 2 Unts 3 Network structure 4 Snglelayer
More information1 Convex Optimization
Convex Optmzaton We wll consder convex optmzaton problems. Namely, mnmzaton problems where the objectve s convex (we assume no constrants for now). Such problems often arse n machne learnng. For example,
More informationSupport Vector Machines
Separatng boundary, defned by w Support Vector Machnes CISC 5800 Professor Danel Leeds Separatng hyperplane splts class 0 and class 1 Plane s defned by lne w perpendcular to plan Is data pont x n class
More informationNon-linear Canonical Correlation Analysis Using a RBF Network
ESANN' proceedngs - European Smposum on Artfcal Neural Networks Bruges (Belgum), 4-6 Aprl, d-sde publ., ISBN -97--, pp. 57-5 Non-lnear Canoncal Correlaton Analss Usng a RBF Network Sukhbnder Kumar, Elane
More informationMultilayer neural networks
Lecture Multlayer neural networks Mlos Hauskrecht mlos@cs.ptt.edu 5329 Sennott Square Mdterm exam Mdterm Monday, March 2, 205 In-class (75 mnutes) closed book materal covered by February 25, 205 Multlayer
More informationLogistic Regression. CAP 5610: Machine Learning Instructor: Guo-Jun QI
Logstc Regresson CAP 561: achne Learnng Instructor: Guo-Jun QI Bayes Classfer: A Generatve model odel the posteror dstrbuton P(Y X) Estmate class-condtonal dstrbuton P(X Y) for each Y Estmate pror dstrbuton
More informationOther NN Models. Reinforcement learning (RL) Probabilistic neural networks
Other NN Models Renforcement learnng (RL) Probablstc neural networks Support vector machne (SVM) Renforcement learnng g( (RL) Basc deas: Supervsed dlearnng: (delta rule, BP) Samples (x, f(x)) to learn
More informationITERATIVE ESTIMATION PROCEDURE FOR GEOSTATISTICAL REGRESSION AND GEOSTATISTICAL KRIGING
ESE 5 ITERATIVE ESTIMATION PROCEDURE FOR GEOSTATISTICAL REGRESSION AND GEOSTATISTICAL KRIGING Gven a geostatstcal regresson odel: k Y () s x () s () s x () s () s, s R wth () unknown () E[ ( s)], s R ()
More informationLecture Slides for. ETHEM ALPAYDIN The MIT Press,
ecture Sldes for ETHEM APAYDI The MIT Press, 00 alpaydn@boun.edu.tr http://www.cpe.boun.edu.tr/~ethe/le Introducton Questons: Assessent of the expected error of a learnng algorth: Is the error rate of
More informationOur focus will be on linear systems. A system is linear if it obeys the principle of superposition and homogenity, i.e.
SSTEM MODELLIN In order to solve a control syste proble, the descrptons of the syste and ts coponents ust be put nto a for sutable for analyss and evaluaton. The followng ethods can be used to odel physcal
More informationSupport Vector Machines
/14/018 Separatng boundary, defned by w Support Vector Machnes CISC 5800 Professor Danel Leeds Separatng hyperplane splts class 0 and class 1 Plane s defned by lne w perpendcular to plan Is data pont x
More informationKernel Methods and SVMs Extension
Kernel Methods and SVMs Extenson The purpose of ths document s to revew materal covered n Machne Learnng 1 Supervsed Learnng regardng support vector machnes (SVMs). Ths document also provdes a general
More informationCOS 511: Theoretical Machine Learning
COS 5: Theoretcal Machne Learnng Lecturer: Rob Schapre Lecture #0 Scrbe: José Sões Ferrera March 06, 203 In the last lecture the concept of Radeacher coplexty was ntroduced, wth the goal of showng that
More informationAdmin NEURAL NETWORKS. Perceptron learning algorithm. Our Nervous System 10/25/16. Assignment 7. Class 11/22. Schedule for the rest of the semester
0/25/6 Admn Assgnment 7 Class /22 Schedule for the rest of the semester NEURAL NETWORKS Davd Kauchak CS58 Fall 206 Perceptron learnng algorthm Our Nervous System repeat untl convergence (or for some #
More informationDiscriminative classifier: Logistic Regression. CS534-Machine Learning
Dscrmnatve classfer: Logstc Regresson CS534-Machne Learnng robablstc Classfer Gven an nstance, hat does a probablstc classfer do dfferentl compared to, sa, perceptron? It does not drectl predct Instead,
More informationMultigradient for Neural Networks for Equalizers 1
Multgradent for Neural Netorks for Equalzers 1 Chulhee ee, Jnook Go and Heeyoung Km Department of Electrcal and Electronc Engneerng Yonse Unversty 134 Shnchon-Dong, Seodaemun-Ku, Seoul 1-749, Korea ABSTRACT
More informationMultilayer Perceptron (MLP)
Multlayer Perceptron (MLP) Seungjn Cho Department of Computer Scence and Engneerng Pohang Unversty of Scence and Technology 77 Cheongam-ro, Nam-gu, Pohang 37673, Korea seungjn@postech.ac.kr 1 / 20 Outlne
More informationLeast Squares Fitting of Data
Least Squares Fttng of Data Davd Eberly Geoetrc Tools, LLC http://www.geoetrctools.co/ Copyrght c 1998-2014. All Rghts Reserved. Created: July 15, 1999 Last Modfed: February 9, 2008 Contents 1 Lnear Fttng
More informationDetermination of the Confidence Level of PSD Estimation with Given D.O.F. Based on WELCH Algorithm
Internatonal Conference on Inforaton Technology and Manageent Innovaton (ICITMI 05) Deternaton of the Confdence Level of PSD Estaton wth Gven D.O.F. Based on WELCH Algorth Xue-wang Zhu, *, S-jan Zhang
More informationCS 3710: Visual Recognition Classification and Detection. Adriana Kovashka Department of Computer Science January 13, 2015
CS 3710: Vsual Recognton Classfcaton and Detecton Adrana Kovashka Department of Computer Scence January 13, 2015 Plan for Today Vsual recognton bascs part 2: Classfcaton and detecton Adrana s research
More informationLeast Squares Fitting of Data
Least Squares Fttng of Data Davd Eberly Geoetrc Tools, LLC http://www.geoetrctools.co/ Copyrght c 1998-2015. All Rghts Reserved. Created: July 15, 1999 Last Modfed: January 5, 2015 Contents 1 Lnear Fttng
More informationLecture 3. Camera Models 2 & Camera Calibration. Professor Silvio Savarese Computational Vision and Geometry Lab. 13- Jan- 15.
Lecture Caera Models Caera Calbraton rofessor Slvo Savarese Coputatonal Vson and Geoetry Lab Slvo Savarese Lecture - - Jan- 5 Lecture Caera Models Caera Calbraton Recap of caera odels Caera calbraton proble
More informationMATH 567: Mathematical Techniques in Data Science Lab 8
1/14 MATH 567: Mathematcal Technques n Data Scence Lab 8 Domnque Gullot Departments of Mathematcal Scences Unversty of Delaware Aprl 11, 2017 Recall We have: a (2) 1 = f(w (1) 11 x 1 + W (1) 12 x 2 + W
More informationXII.3 The EM (Expectation-Maximization) Algorithm
XII.3 The EM (Expectaton-Maxzaton) Algorth Toshnor Munaata 3/7/06 The EM algorth s a technque to deal wth varous types of ncoplete data or hdden varables. It can be appled to a wde range of learnng probles
More informationDepartment of Quantitative Methods & Information Systems. Time Series and Their Components QMIS 320. Chapter 6
Department of Quanttatve Methods & Informaton Systems Tme Seres and Ther Components QMIS 30 Chapter 6 Fall 00 Dr. Mohammad Zanal These sldes were modfed from ther orgnal source for educatonal purpose only.
More informationLogistic Regression Maximum Likelihood Estimation
Harvard-MIT Dvson of Health Scences and Technology HST.951J: Medcal Decson Support, Fall 2005 Instructors: Professor Lucla Ohno-Machado and Professor Staal Vnterbo 6.873/HST.951 Medcal Decson Support Fall
More informationADVANCED MACHINE LEARNING ADVANCED MACHINE LEARNING
1 ADVANCED ACHINE LEARNING ADVANCED ACHINE LEARNING Non-lnear regresson technques 2 ADVANCED ACHINE LEARNING Regresson: Prncple N ap N-dm. nput x to a contnuous output y. Learn a functon of the type: N
More informationLinear Feature Engineering 11
Lnear Feature Engneerng 11 2 Least-Squares 2.1 Smple least-squares Consder the followng dataset. We have a bunch of nputs x and correspondng outputs y. The partcular values n ths dataset are x y 0.23 0.19
More informationKernels in Support Vector Machines. Based on lectures of Martin Law, University of Michigan
Kernels n Support Vector Machnes Based on lectures of Martn Law, Unversty of Mchgan Non Lnear separable problems AND OR NOT() The XOR problem cannot be solved wth a perceptron. XOR Per Lug Martell - Systems
More informationSpectral method for fractional quadratic Riccati differential equation
Journal of Aled Matheatcs & Bonforatcs vol2 no3 212 85-97 ISSN: 1792-662 (rnt) 1792-6939 (onlne) Scenress Ltd 212 Sectral ethod for fractonal quadratc Rccat dfferental equaton Rostay 1 K Kar 2 L Gharacheh
More informationEfficient Weather Forecasting using Artificial Neural Network as Function Approximator
Effcent Weather Forecastng usng Artfcal Neural Network as Functon Approxmator I. El-Fegh, Z. Zuba and S. Abozgaya Abstract Forecastng s the referred to as the process of estmaton n unknown stuatons. Weather
More information1 Review From Last Time
COS 5: Foundatons of Machne Learnng Rob Schapre Lecture #8 Scrbe: Monrul I Sharf Aprl 0, 2003 Revew Fro Last Te Last te, we were talkng about how to odel dstrbutons, and we had ths setup: Gven - exaples
More informationStatistics for Economics & Business
Statstcs for Economcs & Busness Smple Lnear Regresson Learnng Objectves In ths chapter, you learn: How to use regresson analyss to predct the value of a dependent varable based on an ndependent varable
More informationRadial-Basis Function Networks
Radal-Bass uncton Networs v.0 March 00 Mchel Verleysen Radal-Bass uncton Networs - Radal-Bass uncton Networs p Orgn: Cover s theorem p Interpolaton problem p Regularzaton theory p Generalzed RBN p Unversal
More informationDesigning Fuzzy Time Series Model Using Generalized Wang s Method and Its application to Forecasting Interest Rate of Bank Indonesia Certificate
The Frst Internatonal Senar on Scence and Technology, Islac Unversty of Indonesa, 4-5 January 009. Desgnng Fuzzy Te Seres odel Usng Generalzed Wang s ethod and Its applcaton to Forecastng Interest Rate
More informationCIS526: Machine Learning Lecture 3 (Sept 16, 2003) Linear Regression. Preparation help: Xiaoying Huang. x 1 θ 1 output... θ M x M
CIS56: achne Learnng Lecture 3 (Sept 6, 003) Preparaton help: Xaoyng Huang Lnear Regresson Lnear regresson can be represented by a functonal form: f(; θ) = θ 0 0 +θ + + θ = θ = 0 ote: 0 s a dummy attrbute
More informationDr. Shalabh Department of Mathematics and Statistics Indian Institute of Technology Kanpur
Analyss of Varance and Desgn of Exerments-I MODULE III LECTURE - 2 EXPERIMENTAL DESIGN MODELS Dr. Shalabh Deartment of Mathematcs and Statstcs Indan Insttute of Technology Kanur 2 We consder the models
More informationRecap: the SVM problem
Machne Learnng 0-70/5-78 78 Fall 0 Advanced topcs n Ma-Margn Margn Learnng Erc Xng Lecture 0 Noveber 0 Erc Xng @ CMU 006-00 Recap: the SVM proble We solve the follong constraned opt proble: a s.t. J 0
More informationStudy of Classification Methods Based on Three Learning Criteria and Two Basis Functions
Study of Classfcaton Methods Based on hree Learnng Crtera and wo Bass Functons Jae Kyu Suhr Abstract - hs paper nvestgates several classfcaton ethods based on the three learnng crtera and two bass functons.
More informationWhat is LP? LP is an optimization technique that allocates limited resources among competing activities in the best possible manner.
(C) 998 Gerald B Sheblé, all rghts reserved Lnear Prograng Introducton Contents I. What s LP? II. LP Theor III. The Splex Method IV. Refneents to the Splex Method What s LP? LP s an optzaton technque that
More informationGenerative classification models
CS 675 Intro to Machne Learnng Lecture Generatve classfcaton models Mlos Hauskrecht mlos@cs.ptt.edu 539 Sennott Square Data: D { d, d,.., dn} d, Classfcaton represents a dscrete class value Goal: learn
More informationOutline. Review Numerical Approach. Schedule for April and May. Review Simple Methods. Review Notation and Order
Sstes of Ordnar Dfferental Equatons Aprl, Solvng Sstes of Ordnar Dfferental Equatons Larr Caretto Mecancal Engneerng 9 Nuercal Analss of Engneerng Sstes Aprl, Outlne Revew bascs of nuercal solutons of
More informationDr. Shalabh Department of Mathematics and Statistics Indian Institute of Technology Kanpur
Analyss of Varance and Desgn of Experment-I MODULE VII LECTURE - 3 ANALYSIS OF COVARIANCE Dr Shalabh Department of Mathematcs and Statstcs Indan Insttute of Technology Kanpur Any scentfc experment s performed
More informationON WEIGHTED ESTIMATION IN LINEAR REGRESSION IN THE PRESENCE OF PARAMETER UNCERTAINTY
Econoetrcs orkng Paper EP7 ISSN 485-644 Departent of Econocs ON EIGTED ESTIMATION IN LINEAR REGRESSION IN TE PRESENCE OF PARAMETER UNCERTAINTY udth A Clarke Departent of Econocs, Unversty of Vctora Vctora,
More informationLecture 10 Support Vector Machines II
Lecture 10 Support Vector Machnes II 22 February 2016 Taylor B. Arnold Yale Statstcs STAT 365/665 1/28 Notes: Problem 3 s posted and due ths upcomng Frday There was an early bug n the fake-test data; fxed
More informationUsing T.O.M to Estimate Parameter of distributions that have not Single Exponential Family
IOSR Journal of Mathematcs IOSR-JM) ISSN: 2278-5728. Volume 3, Issue 3 Sep-Oct. 202), PP 44-48 www.osrjournals.org Usng T.O.M to Estmate Parameter of dstrbutons that have not Sngle Exponental Famly Jubran
More informationMultipoint Analysis for Sibling Pairs. Biostatistics 666 Lecture 18
Multpont Analyss for Sblng ars Bostatstcs 666 Lecture 8 revously Lnkage analyss wth pars of ndvduals Non-paraetrc BS Methods Maxu Lkelhood BD Based Method ossble Trangle Constrant AS Methods Covered So
More informationOnline Classification: Perceptron and Winnow
E0 370 Statstcal Learnng Theory Lecture 18 Nov 8, 011 Onlne Classfcaton: Perceptron and Wnnow Lecturer: Shvan Agarwal Scrbe: Shvan Agarwal 1 Introducton In ths lecture we wll start to study the onlne learnng
More informationComputational and Statistical Learning theory Assignment 4
Coputatonal and Statstcal Learnng theory Assgnent 4 Due: March 2nd Eal solutons to : karthk at ttc dot edu Notatons/Defntons Recall the defnton of saple based Radeacher coplexty : [ ] R S F) := E ɛ {±}
More informationSolving Fuzzy Linear Programming Problem With Fuzzy Relational Equation Constraint
Intern. J. Fuzz Maeatcal Archve Vol., 0, -0 ISSN: 0 (P, 0 0 (onlne Publshed on 0 Septeber 0 www.researchasc.org Internatonal Journal of Solvng Fuzz Lnear Prograng Proble W Fuzz Relatonal Equaton Constrant
More informationLecture 3: Dual problems and Kernels
Lecture 3: Dual problems and Kernels C4B Machne Learnng Hlary 211 A. Zsserman Prmal and dual forms Lnear separablty revsted Feature mappng Kernels for SVMs Kernel trck requrements radal bass functons SVM
More informationClassification learning II
Lecture 8 Classfcaton learnng II Mlos Hauskrecht mlos@cs.ptt.edu 539 Sennott Square Logstc regresson model Defnes a lnear decson boundar Dscrmnant functons: g g g g here g z / e z f, g g - s a logstc functon
More informationEEE 241: Linear Systems
EEE : Lnear Systems Summary #: Backpropagaton BACKPROPAGATION The perceptron rule as well as the Wdrow Hoff learnng were desgned to tran sngle layer networks. They suffer from the same dsadvantage: they
More informationStatistics for Business and Economics
Statstcs for Busness and Economcs Chapter 11 Smple Regresson Copyrght 010 Pearson Educaton, Inc. Publshng as Prentce Hall Ch. 11-1 11.1 Overvew of Lnear Models n An equaton can be ft to show the best lnear
More informationXiangwen Li. March 8th and March 13th, 2001
CS49I Approxaton Algorths The Vertex-Cover Proble Lecture Notes Xangwen L March 8th and March 3th, 00 Absolute Approxaton Gven an optzaton proble P, an algorth A s an approxaton algorth for P f, for an
More informationPerceptual Organization (IV)
Perceptual Organzaton IV Introducton to Coputatonal and Bologcal Vson CS 0--56 Coputer Scence Departent BGU Ohad Ben-Shahar Segentaton Segentaton as parttonng Gven: I - a set of age pxels H a regon hoogenety
More informationOn the Eigenspectrum of the Gram Matrix and the Generalisation Error of Kernel PCA (Shawe-Taylor, et al. 2005) Ameet Talwalkar 02/13/07
On the Egenspectru of the Gra Matr and the Generalsaton Error of Kernel PCA Shawe-aylor, et al. 005 Aeet alwalar 0/3/07 Outlne Bacground Motvaton PCA, MDS Isoap Kernel PCA Generalsaton Error of Kernel
More informationGadjah Mada University, Indonesia. Yogyakarta State University, Indonesia Karangmalang Yogyakarta 55281
Reducng Fuzzy Relatons of Fuzzy Te Seres odel Usng QR Factorzaton ethod and Its Applcaton to Forecastng Interest Rate of Bank Indonesa Certfcate Agus aan Abad Subanar Wdodo 3 Sasubar Saleh 4 Ph.D Student
More informationNegative Binomial Regression
STATGRAPHICS Rev. 9/16/2013 Negatve Bnomal Regresson Summary... 1 Data Input... 3 Statstcal Model... 3 Analyss Summary... 4 Analyss Optons... 7 Plot of Ftted Model... 8 Observed Versus Predcted... 10 Predctons...
More informationOutline. EM Algorithm and its Applications. K-Means Classifier. K-Means Classifier (Cont.) Introduction of EM K-Means EM EM Applications.
EM Algorthm and ts Alcatons Y L Deartment of omuter Scence and Engneerng Unversty of Washngton utlne Introducton of EM K-Means EM EM Alcatons Image Segmentaton usng EM bect lass Recognton n BIR olor lusterng
More informationHidden Markov Model Cheat Sheet
Hdden Markov Model Cheat Sheet (GIT ID: dc2f391536d67ed5847290d5250d4baae103487e) Ths document s a cheat sheet on Hdden Markov Models (HMMs). It resembles lecture notes, excet that t cuts to the chase
More informationROC ANALYSIS FOR PREDICTIONS MADE BY PROBABILISTIC CLASSIFIERS
Proceedngs of the Fourth Internatonal Conference on Machne earnng and Cybernetcs, Guangzhou, 8- August 005 ROC ANAYSIS FOR PREDICTIONS MADE BY PROBABIISTIC CASSIFIERS ZENG-CHANG QIN Artfcal Intellgence
More informationSlobodan Lakić. Communicated by R. Van Keer
Serdca Math. J. 21 (1995), 335-344 AN ITERATIVE METHOD FOR THE MATRIX PRINCIPAL n-th ROOT Slobodan Lakć Councated by R. Van Keer In ths paper we gve an teratve ethod to copute the prncpal n-th root and
More informationC4B Machine Learning Answers II. = σ(z) (1 σ(z)) 1 1 e z. e z = σ(1 σ) (1 + e z )
C4B Machne Learnng Answers II.(a) Show that for the logstc sgmod functon dσ(z) dz = σ(z) ( σ(z)) A. Zsserman, Hlary Term 20 Start from the defnton of σ(z) Note that Then σ(z) = σ = dσ(z) dz = + e z e z
More informationLINEAR REGRESSION ANALYSIS. MODULE IX Lecture Multicollinearity
LINEAR REGRESSION ANALYSIS MODULE IX Lecture - 31 Multcollnearty Dr. Shalabh Department of Mathematcs and Statstcs Indan Insttute of Technology Kanpur 6. Rdge regresson The OLSE s the best lnear unbased
More informationA goodness-of-fit measure for a system-of-equations model
Songklanakarn J. Sc. Technol. 32 (5), 519-525, Sep. - Oct. 2010 Orgnal Artcle A goodness-of-ft easure for a syste-of-equatons odel Jrawan Jtthavech* School of Appled Statstcs Natonal Insttute of Developent
More informationMixture of Gaussians Expectation Maximization (EM) Part 2
Mture of Gaussans Eectaton Mamaton EM Part 2 Most of the sldes are due to Chrstoher Bsho BCS Summer School Eeter 2003. The rest of the sldes are based on lecture notes by A. Ng Lmtatons of K-means Hard
More information